
PostgreSQL - eRServer Installation Guide


Project:
PostgreSQL Enterprise Replication Server

Abstract:

This guide describes the steps required in order to install and configure eRServer 1.2 in a master to one or multi-slave configuration.

Author:

Justin Clift - Digital Distribution Global Financial Services Pty. Ltd.


Approval:

















Authorisation:
















Document Reference:	ERSRV-INST
Date: 				18th November 2002


Document Control
 
Amendment Record

Issue
Date
Author
Comments
1.0
6th October 2002
Justin Clift
Original version
1.1
8th October 2002
Justin Clift
Updated path information and graphic.
1.2
12th October 2002
Justin Clift
Incorporated suggestions from PgSQL and Afilias, added the information about PL/pgSQL.
1.3
18th November 2002
Justin Clift
Fixed simple typo(s)

The latest version of this document supersedes all other previous issues.




ABBREVIATIONS
 
AUP
Acceptable Use Policy
OS
Operating System
PgSQL
PostgreSQL, Inc.
TCP/IP
Transport Control Protocol/Internet Protocol


Index
1 INTRODUCTION	4
1.1 GENERAL	4
1.2 USEFUL INTERNET LINKS	4
2 REQUIREMENTS	4
3 OVERVIEW	5
4 INSTALL THE ERSERVER SOFTWARE	6
4.1 CREATE A UNIX ACCOUNT FOR ERSERVER	6
4.2 COMPILE AND INSTALL ERSERVER	6
5 INSTALLATION OF ERSERVER FOR THE MASTER DATABASE	7
5.1 INSTALLATION OF THE ERSERVER LIBRARIES ON THE MASTER DATABASE SERVER	7
5.2 ADD THE PL/PGSQL LANGUAGE TO YOUR MASTER DATABASE	7
5.3 CREATION OF THE ERSERVER FUNCTIONS IN THE MASTER DATABASE	7
5.4 CREATION OF THE REPLICATION INFORMATION TABLES IN THE MASTER DATABASE	8
5.5 CONFIGURING THE MASTER DATABASE ENTRY IN THE CONFIGURATION FILE	8
6 INSTALLATION OF ERSERVER ON A SLAVE DATABASE	10
6.1 CREATION OF THE REPLICATION INFORMATION TABLES IN THE SLAVE DATABASE(S)	10
6.2 UPDATE THE REPLICATION.CFG FILE TO REFLECT THE SETTINGS NEEDED FOR THE SLAVE DATABASE(S)	10
6.3 CONFIGURING MULTIPLE SLAVE DATABASES	11
7 ADDING TABLES TO THE REPLICATION LIST	12
7.1 ADDING TABLES IN THE MASTER DATABASE	12
7.2 ADDING TABLES IN THE SLAVE DATABASES	12
7.3 POINTS TO REMEMBER	12
8 SECURING THE ERSERVER INSTALLATION	13
8.1 ENSURE THE REPLICATION.CFG FILE IS ONLY READABLE BY THE ERSERVER USER	13
8.2 ENSURE THE LOG FILES ARE ONLY READABLE BY THE ERSERVER USER	13
9 STARTING AND STOPPING ERSERVER	14
9.1 STARTING ERSERVER	14
9.2 STOPPING ERSERVER	14
10 FINAL CHECKLIST	15
10.1 ENSURE DEBUG AND VERBOSE MODE ARE TURNED OFF IN THE REPLICATION.CFG FILE	15


 
1 
 Introduction
1.1  General
This document details the installation and configuration of PostgreSQL Inc.'s Enterprise Replication Server (eRServer).  It is geared to an intended audience of System Installers, Unix Administrators and Security Managers / Officers within PostgreSQL, Inc. project and technical support teams.


1.2  Useful Internet Links
eRServer			http://www.erserver.com

PostgreSQL		http://www.postgresql.org

PostgreSQL, Inc.	http://www.pgsql.com



2  REQUIREMENTS

1. A supported Operating System (Solaris, FreeBSD, Linux)
2. Sun JDK 1.2.x, or 1.3.1
3. Perl 5.005_03 or greater
4. eRServer 1.2
5. GNU gcc 2.95.3 or greater
6. GNU make 3.79.1 or greater
7. PostgreSQL Superuser access to the master database(s)



3 
 OVERVIEW

This section gives an overview of eRServer in an n-tier environment.

This diagram illustrates that those Data processors needing write access to the data use the Master database, whilst the bulk of database access (read only transactions and aggregate queries) access the Slave databases, updated by eRServer.

In addition to the disaster recovery and business continuity capabilities that replication provides,
eRServer enables the offloading of database activity from a single PostgreSQL database server, spreading it across many.

Other supported functions for a 
Multi-Slave setup include creating 
Test Environments for live loading of real data, and to support specialized business applications such as EIS/DSS Servers.














4 
 INSTALL THE ERSERVER SOFTWARE

4.1  Create a unix account for eRServer

For maximum flexibility and security we recommend you create a unix "erserver" user and unix "erserver" group for the eRServer software to be controlled by.

For Solaris and Linux:

# groupadd erserver
# useradd -g erserver

For FreeBSD:

# pw groupadd erserver
# adduser -s erserver

4.2  Compile and install eRServer

Extract the eRServer 1.2 tar file in a temporary installation directory.

$ tar xfz erserver12.tar.gz

Enter the extracted directory:

$ cd erserver

By default the target installation directory is /usr/opt/erserver.  If you wish to change this then edit the file Makefile.global before running the make command and edit the installation path in the "prefix :=" line.

$ vi Makefile.global

Compile the source code:

$ make

Install the eRServer software in the target installation directory:

$ su
# make install
# exit
$
5  INSTALLATION OF ERSERVER FOR THE MASTER DATABASE
5.1  Installation of the eRServer libraries on the master database server
There are two files, erserver.so and pte.so, that must be installed on the master database server.  These contain code that will be called by the PostgreSQL trigger functions installed in the next step.

In an environment using a failover solution, these libraries must also be installed on all servers that could become a master database server in the event of such a failover.

You need to take note of the path to where you install these files, as it is required for the following step.

$ mkdir -p /usr/opt/erserver/lib
$ cp erserver.so pte.so /usr/opt/erserver/lib


5.2  Add the PL/pgSQL language to your master database
The PostgreSQL functions needed by eRServer are written in the procedural language PL/pgSQL.

You need to enable this language for each master database by using this command:

$ createlang plpgsql masterdb

An alternative method is to enable PL/pgSQL for the special template1 database, and any subsequent databases (including the master database) created after this will automatically have it enabled.

$ createlang plpgsql template1


5.3  Creation of the eRServer functions in the master database
Next you must create the PostgreSQL trigger functions used by eRServer to keep the replication information updated in the master database.

As with the eRServer libraries, this procedure must be completed for every server that could become a master database server in a failover situation.

You must connect as the PostgreSQL database superuser for this to be successful, and pass this script the path to the remote erserver.so and pte.so files installed by the previous step.

$ cd /usr/opt/erserver/bin
$ ./ers_mastercreatefunctions --host=masterhost --user=postgres --password=pgpassword \
        --remotepath=/usr/opt/erserver/lib masterdb 


5.4  Creation of the Replication information tables in the master database
Each master database uses several tables to hold and co-ordinate the replication.  You should set these up by running the ers_mastercreatetables command as the owner of the database in question, although it doesn't have to be the PostgreSQL superuser.

$ cd /usr/opt/erserver/bin
$ ./ers_mastercreatetables --host=masterhost --user=masteruser --password=masterpw masterdb


5.5  Configuring the master database entry in the configuration file
You now need to ensure the JDBC connection information for the master database is correct in the replication.cfg configuration file.  This file is in the "conf" subdirectory of your eRServer installation.

The entries you need to update are:

replic.master.ConnectionPoolSize=3

This controls how many threads are started, one is needed per slave database.  More will be added as needed up until the maximum defined by ConnectionPoolMax.


replic.master.ConnectionPoolMax=4

This controls the maximum number of threads allowed to be running.  Make sure you have enough to service all of your slave databases.


replic.master.JDBCConnectionURL=jdbc:postgresql://192.168.10.1:5432/masterdb

This controls the hostname (or IP address) of the master database, the TCP number port it is listening on, and the name of the master database.


replic.master.user=masteruser

This specifies the PostgreSQL database username to connect to the master database as.


replic.master.password=masterpw

This gives the password used to connect to the master database.

The eRServer daemon reads this information once at startup, so if you change this information later on you will need to restart the daemon.



















6 
 INSTALLATION OF ERSERVER ON A SLAVE DATABASE
This needs to be done for each slave database.


6.1  Creation of the Replication information tables in the slave database(s)

Each slave database uses several tables to hold and co-ordinate the replication.  You should set these up by running the ers_slavecreatetables command as the owner of the database in question.

$ cd /usr/opt/erserver/bin
$ ./ers_slavecreatetables --host=slavehost --user=slaveuser --password=slavepw slavedb1


6.2  Update the replication.cfg file to reflect the settings needed for the slave database(s)
You now need to ensure the JDBC connection information for the slave database is correct in the replication.cfg configuration file.  This file is in the "conf" subdirectory of your eRServer installation.

replic.server.ReplicInterval=5

This controls how many seconds of delay are between replication updates.  This should be tuned depending on expected server load.


replic.slave.JDBCConnectionURL=jdbc:postgresql://127.0.0.1:5432/slavedb1

This controls the hostname (or IP address) of the slave database, the TCP number port it is listening on, and the name of the slave database.


replic.slave.user=slaveuser

This specifies the PostgreSQL database username to connect to the slave database as.


replic.slave.password=slavepw

This gives the password used to connect to the slave database.


replic.slave.ConnectionPoolSize=1
replic.slave.ConnectionPoolMax= 3
replic.slave.ConnectionUseCount=1000
replic.slave.ConnectionTimeout=2

Don't alter these values unless fine tuning eRServer and you really understand the consequences.


6.3  Configuring multiple slave databases
With each additional slave database, you will need to append additional comma separated entries to all of these lines otherwise the database will function unreliably.

For example, if you have three slave databases then the slave configuration options in the replication.cfg will look similar to this:

replic.server.ReplicInterval=5,5,5
replic.slave.JDBCConnectionURL=jdbc:postgresql://127.0.0.1:5432/slavedb1, jdbc:postgresql://127.0.0.2:5432/slavedb2, jdbc:postgresql://127.0.0.3:5432/slavedb3
replic.slave.user=slaveuser,slave1user,slave2user
replic.slave.password=slavepw,slave1pw,slave2pw
replic.slave.ConnectionPoolSize=1,1,1
replic.slave.ConnectionPoolMax= 3,3,3
replic.slave.ConnectionUseCount=1000,1000,1000
replic.slave.ConnectionTimeout=2,2,2











7 
 ADDING TABLES TO THE REPLICATION LIST

7.1  Adding tables from the master database
Adding tables to the replication list of the master database is done with the ers_masteraddtable command.  You give it the correct connection details for the server and database, in addition to the name of the table and an integer based column in the table.

$ ers_masteraddtable --host=masterhost --user=masteruser --password=masterpw masterdb table key_column


7.2  Adding tables for the slave databases
Adding tables to the replication list of the slave database is done with the ers_slaveaddtable command.  You give it the correct connection details for the server and database, in addition to the name of the table and an integer based column in the table.

$ ers_slaveaddtable --host=slavehost --user=slaveuser --password=slavepw slavedb table key_column


7.3  Points to remember
* Every table for replication must have an integer based key.

* All tables to be replicated must be added to both the master and slaves.

* Sequences in the master database aren't updated on the slave database(s).

If you have tables in the master database that use Sequences for number generation, the corresponding Sequences in the slave database(s) will remain unchanged regardless of how much data is replicated.

In a failover situation where a slave database needs to become the master database, the Sequences in the slave database have to be updated to current values its operation as a master begins, otherwise duplicate sequence numbers will occur and insertions in the database will fail.

This is done through using the setval() and max() commands on the slave database.






8 
 SECURING THE ERSERVER INSTALLATION
8.1  Ensure the replication.cfg file is only readable by the erserver user
Because the replication.cfg file contains the passwords used to connect to your remote master and slave databases, you need to take care that it's only readable by the erserver user.

$ chmod 600 conf/replication.cfg


8.2  Ensure the log files are only readable by the erserver user
Because sensitive information can sometimes be present in the eRServer log files, you need to take care that they too are only readable by the erserver user.

$ chmod 700 log
$ chmod 600 log/*


9 
 STARTING AND STOPPING ERSERVER
9.1  Starting eRServer
eRServer requires the Sun JDK 1.2.x or 1.3.x, and is recommended to be run on the same server as the master database.

You start it as the erserver user, as follows:

$ cd [erserver base installation directory]/bin
$ ./erserver.sh start


9.2  Stopping eRServer
You stop eRServer as the erserver user, as follows:

$ cd [erserver base installation directory]/bin
$ ./erserver.sh stop





10 
 FINAL CHECKLIST
10.1  Ensure debug and verbose mode are turned off in the replication.cfg file

Both of these modes have a performance hit when used, although the verbose mode is negligible.

An additional reason to turn them off in a production environment is because of the large amount of log files they generate.



TUNING ERSERVER INSTALLATIONS

- Have fsync() on for masters, but turn fsync() off for [read-only] slaves.  This ensures data integrity where it counts, and speed where it's needed.

PostgreSQL, Inc. Confidential Document
Commercially Sensitive
Do Not Distribute

Copyright (c) 2002 PostgreSQL, Inc.
Page 3 of 1


PostgreSQL, Inc. Confidential Document
Commercially Sensitive
Do Not Distribute

Copyright (c) 2002 PostgreSQL, Inc.
Page 1 of 1


PostgreSQL, Inc. Confidential Document
Commercially Sensitive
Do Not Distribute

Copyright (c) 2002 PostgreSQL, Inc.
Page 8 of 1


PostgreSQL, Inc. Confidential Document
Commercially Sensitive
Do Not Distribute

Copyright (c) 2002 PostgreSQL, Inc.
Page 4 of 1


PostgreSQL, Inc. Confidential Document
Commercially Sensitive
Do Not Distribute

Copyright (c) 2002 PostgreSQL, Inc.
Page 12 of 1

